Skip to content

Arm backend: Add manifest validation and add Backwards compabillety testing#18813

Open
Sebastian-Larsson wants to merge 3 commits intopytorch:mainfrom
Sebastian-Larsson:validate_manifest
Open

Arm backend: Add manifest validation and add Backwards compabillety testing#18813
Sebastian-Larsson wants to merge 3 commits intopytorch:mainfrom
Sebastian-Larsson:validate_manifest

Conversation

@Sebastian-Larsson
Copy link
Copy Markdown
Collaborator

@Sebastian-Larsson Sebastian-Larsson commented Apr 10, 2026 •
edited by pytorch-bot Bot
Loading

@Sebastian-Larsson Sebastian-Larsson added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk release notes: arm Changes to the ARM backend delegate labels Apr 10, 2026
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Apr 10, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18813

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 1 New Failure, 4 Unrelated Failures

As of commit a799b2d with merge base 36e8ed9 (image):

NEW FAILURE - The following job has failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 10, 2026
@zingo
Copy link
Copy Markdown
Collaborator

zingo commented Apr 13, 2026

Hi @digantdesai we tried to handle the buck2 files with this, but you might want to take a peek anyway.

@zingo zingo changed the title Arm backend: Add manifest validation and add BC testing Arm backend: Add manifest validation and add Backwards Comp. testing Apr 13, 2026
@zingo zingo changed the title Arm backend: Add manifest validation and add Backwards Comp. testing Arm backend: Add manifest validation and add Backwards compabillety testing Apr 13, 2026
@Sebastian-Larsson
Arm backend: Add public API manifest validator
9cb631d 
Add a validator script that compares one public API manifest against the
current backends/arm API using exact symbol, kind, and signature
matching.

Add regression tests for manifest drift, and run the validator from the
Arm pre-push hook on every push.

Change-Id: I4765510b8a2f8c5aa09717fc8287d976c770897d
Signed-off-by: Sebastian Larsson <sebastian.larsson@arm.com>
@Sebastian-Larsson
Arm backend: Add public API validation and BC tests to CI
a227111 
Introduce BC tests and add those as well as the validation of the public
API manifest to CI in the pull job.

Change-Id: I1815223a13109e225f2b9e0f9914b504799db2ae
Signed-off-by: Sebastian Larsson <sebastian.larsson@arm.com>
@Sebastian-Larsson
Arm backend: Flatten nested Python manifest tables
a799b2d 
Flatten nested TOML tables when reading Python public API
manifest entries so validation also checks nested symbols.

Add regression coverage that shows nested entries are parsed
and that missing nested symbols are reported by validation.

Signed-off-by: Sebastian Larsson <sebastian.larsson@arm.com>
Change-Id: Ic04883d07647b3b6f4ed9b86c68d969dcbd90ade
@zingo
Copy link
Copy Markdown
Collaborator

zingo commented Apr 14, 2026

@digantdesai this adds some files and update buck2 stuff so you might want to check this :)

@GregoryComer
Copy link
Copy Markdown
Member

GregoryComer commented Apr 20, 2026

Thanks. Buck CI is running internally now and I'll report back if there's anything issues.

@meta-codesync
Copy link
Copy Markdown
Contributor

meta-codesync Bot commented Apr 20, 2026

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this in D101685482.

digantdesai
digantdesai reviewed Apr 20, 2026
View reviewed changes
Comment thread backends/arm/public_api_manifests/api_manifest_running.toml
#
# This file is generated by
# backends/arm/scripts/generate_public_api_manifest.py
# backends/arm/scripts/public_api_manifest/generate_public_api_manifest.py
Copy link
Copy Markdown
Contributor

@digantdesai digantdesai Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is not the PR introducing this API checker, but two questions,
(1) why not rely on unit-tests to enforce the actual behavior, beyond API BC
(2) why not use Griffe or something similar to do this for you?

digantdesai
digantdesai approved these changes Apr 20, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@digantdesai digantdesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok with this PR, but left a couple of question for the original motivation (sorry I was out for a few months)

@GregoryComer
Copy link
Copy Markdown
Member

GregoryComer commented Apr 20, 2026

The internal buck build is complaining about the following: error: fail: src files must be named test_.py, _test.py, or conftest.py: public_api_bc/ethosu_flow.py. Seems like an odd constraint, but would it be possible to rename this file? CC @digantdesai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@digantdesai digantdesai digantdesai approved these changes

Assignees

No one assigned

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Sebastian-Larsson @zingo @GregoryComer @digantdesai